The use and impact of digital COVID-19 tracking in adult social care: a prospective cohort study of care homes in Greater Manchester

Background To support proactive care during the coronavirus pandemic, a digital COVID-19 symptom tracker was deployed in Greater Manchester (UK) care homes. This study aimed to understand what factors were associated with the post-uptake use of the tracker and whether the tracker had any effects in controlling the spread of COVID-19. Methods Daily data on COVID-19, tracker uptake and use, and other key indicators such as staffing levels, the number of staff self-isolating, availability of personal protective equipment, bed occupancy levels, and any problems in accepting new residents were analysed for 547 care homes across Greater Manchester for the period April 2020 to April 2021. Differences in tracker use across local authorities, types of care homes, and over time were assessed using correlated effects logistic regressions. Differences in numbers of COVID-19 cases in homes adopting versus not adopting the tracker were compared via event design difference-in-difference estimations. Results Homes adopting the tracker used it on 44% of days post-adoption. Use decreased by 88% after one year of uptake (odds ratio 0.12; 95% confidence interval 0.06–0.28). Use was highest in the locality initiating the project (odds ratio 31.73; 95% CI 3.76–268.05). Care homes owned by a chain had lower use (odds ratio 0.30; 95% CI 0.14–0.63 versus single ownership care homes), and use was not associated with COVID-19 or staffing levels. Tracker uptake had no impact on controlling COVID-19 spread. Staff self-isolating and local area COVID-19 cases were positively associated with lagged COVID-19 spread in care homes (relative risks 1.29; 1.2–1.4 and 1.05; 1.0–1.1, respectively). Conclusions The use of the COVID-19 symptom tracker in care homes was not maintained except in Locality 1 and did not appear to reduce the COVID-19 spread. COVID-19 cases in care homes were mainly driven by care home local-area COVID-19 cases and infections among the staff members. Digital deterioration trackers should be co-produced with care home staff, and local authorities should provide long-term support in their adoption and use. Supplementary Information The online version contains supplementary material available at 10.1186/s12879-022-07939-6.

An intermediate approach that helps retrieve the effects of the observed time invariant characteristics and relies on a weaker assumption than the random effects model is the correlated random effects model. 1 This model separates within-and between-home effects by including the original variables and their home specific mean values for the variables that vary across homes and over time (such as staffing levels, PPE, and occupied beds). Thus, when our interest was to retrieve the effects of localities and home types, we used correlated random effects regressions (manuscript: Column 2 of Table 2).
Missing data Two of the homes adopted the tracker in the last week of the sample and were dropped from the analysis. Seven care homes were dropped from the correlated random effects regression as those could not be linked to CQC data on homes' characteristics. Rest of the data were complete except for the use indicator, having approximately 4% missing values. This means that missing data were highly unlikely to create any bias in our results. 2 As a further check, the missingness in use indicator was regressed on the observed covariates along with care home fixed effects. None of the covariates were associated with the missing observations in the use indicator after controlling for home fixed effects. This implies that the missing data in the use indicator is missing completely at random (MCAR), and complete case analysis is a valid approach. 3 The results in Table A1 replaced the time since adoption indicators of Table 2 with calendar month indicators and indicators on the number of care homes with the tracker on a given day. The former indicators capture how the tracker use decreases over calendar time, whereas the latter indicators control for the number of care  Notes: *** p<0.01, ** p<0.05, * p<0.1. Cluster robust 95% confidence intervals are in the brackets. The dependent variable is tracker use [=1 if at least one resident is assessed on a given day in a care home, 0 otherwise]. The coefficient of one locality is not reported due to very low uptake. Columns 1 and 2 exclude 7 homes due to missing CQC data. The estimations in Columns 1 and 2 also include a categorical variable on home local area index of multiple deprivation; the coefficients are mostly insignificant and omitted to save space. The results were obtained from logistic regressions. To prevent omitted variables bias Column 2 includes additional variables and their means (means not reported) . Columns 3 and 4 removes the time invariant characteristics of homes by including home fixed effects.
homes on a given day to account for the differences in use that might arise from new homes adopting the tracker. These results also confirmed that tracker use decreased over time, care homes owned by a chain had lower odds of tracker use, and use was significantly higher in Locality 1 compared with the other localities, though the magnitude of the difference is now lower compared with Table 2. Table A2 introduced interactions between the time since adoption and locality indicators to access whether the main results of Table 2 hold after further generalisation of the regression model. would have been the same in the intervention group as in the control group in the absence of the intervention (parallel trends assumption). In practice, one can only test pre-intervention parallel trends to assess the plausibility of this assumption. The DID analyses were conducted with 1) standard Poisson regression, 2) event design Poisson regression, and 3) weighting-based non/semi-parametric DID estimations.
In the standard fixed effects Poisson regression (Columns 3-5 of Table 3 Table   A6, all the DID estimates in the Appendix 4 come from event design. As a further robustness check, we also implemented the non/semi-parametric 7 estimation approach of Callaway and Sant'Anna. 8 This approach allows for treatment heterogeneity across both care homes and over time. With daily data having many homes, it involved estimations of thousands of two-period x two-groups (2x2) treatment effects. To get around this, we aggregated the data to weekly-levels. The 2x2 5 Goodman-Bacon, A. (2021). Difference-in-differences with variation in treatment timing. Journal of Econometrics, 225(2), 254-277. 6 For the difference between the two methods compare Equations 1 and 6 (also a detailed example on page 5) of Clarke, D., and Schythe, K. (2020). Implementing the panel event study. IZA Discussion Paper No. 13524, Institute of Labor Economics (IZA). https://ssrn.com/abstract=3660271. 7 A weak point of any regression based DID analysis is the parametric assumptions, i.e. the fixed effects and the error term is additively separable of the other confounders. Non-parametric estimations do not impose any functional form restrictions.  Notes: *** p<0.01, ** p<0.05, * p<0.1. Cluster robust 95% confidence intervals are in the brackets. The coefficients are incidence rate ratios. All the control variables are included with lag 7 keeping in view the incubation period of COVID-19. Column 3 estimations were run on a matched sub-sample (matching was done on all the characteristics reported in the CQC data given in manuscript Table 1 with nearest 5 neighbours for each treated home). The estimations included 34 pre and 34 post uptake event indicators, but to save space reports only the ones with seven days gap.
intervention effects were then aggregated using time-to-intervention as weights.
Column 1 of Table A4 reports results for crude/unadjusted average treatment effects from this approach, whereas Columns 2 and 3 results are conditional average treatment effects (controlling for pre-intervention covariates).
Missing data 13 care homes were in the database but had no data on  or occupied beds during 2020. The DID estimations included 534 care homes with 127,589 observations (homes x days). Over the 2020 period, 124 care homes had no positive/symptomatic residents, thus they were excluded from the DID estimations with Poisson regression (as the outcomes of such homes are predicted by the home indicator alone in probability-based regressions). Nevertheless, these homes were included in the non/semi-parametric estimations in Table A4. Notes: *** p<0.01, ** p<0.05 * p<0.1. Bootstrap (999 repetitions) 95% confidence intervals are in the brackets. These non/semi-parametric results come from Callaway and Sant'Anna approach on weekly data. Column 2 adjusts for the pre tracker use values of all confounders in Table A3. Column 3 also adjusts for the pre tracker use values of all the variables in manuscript Table 1 including home CQC rating and home area index of multiple deprivation. Adjustment is done through propensity scores re-weighting. The estimations included all pre and post uptake event weeks, but to save space all are not reported.  Notes: *** p<0.01, ** p<0.05, * p<0.1. Cluster robust 95% confidence intervals are in the brackets. These results are for the 4 localities (1-4) that opted the tracker before mid-November 2020. Columns 4 and 5 estimations are run on a matched sub-sample (matching is done on all the characteristics reported in the CQC data given in manuscript Table 1 with nearest neighbours for each treated home). Rest of the details are as in Table A3.